SDSC Boosts Comet’s Capabilities with Seagate Solid State Drives

Published November 10, 2015

The addition of the 800GB (gigabyte) Seagate SAS SSDs will significantly boost Comet’s data analytics capability by expanding its node-local storage capacity for data-intensive workloads. Pairs of the drives will be added to all 72 compute nodes in one rack of Comet, alongside the existing SSDs. This will bring the flash storage in a single node to almost 2TB (terabytes), with total rack capacity at more than 138TB.

The installation process of the Seagate drives began in October under a donation arrangement with SDSC and its Center for Large Scale Data Systems Research, an industry-university collaboration that focuses on issues including “big data” system architectures and software, analytics, and performance, with the goal of understanding the full value that can be extracted from voluminous amounts of data now becoming available to organizations. User access to the drives will begin before January 2016.

Under the partnership, Seagate and SDSC/CLDS are deploying a lightweight framework for extracting metrics suitable for the analysis of data I/O patterns and overall drive performance. These results and other metrics will be used to further develop best practices and reference HPC (high-performance computing) architectures, while leading to more precise analyses of SSD performance in operational HPC workloads.

“The addition of the Seagate solid state drives on Comet continues the work we pioneered in the area of system versatility around flash-based SSDs on our Gordon supercomputer,” said SDSC Director Michael Norman, also principal investigator for the Comet project. “It complements Comet’s other dimensions, such as its fast Lustre parallel file storage systems and large memory nodes. With such a wide range of workflows in both traditional and emerging science domains such as genomics, the greater research community will benefit from these heterogeneous but integrated capabilities.”

The new drives will also extend the abilities of Comet’s upcoming virtualized HPC clusters. “Currently, some virtual machines on Comet can have large disk images that take advantage of the fast local storage on the compute nodes hosting them,” said Rick Wagner, SDSC’s manager of HPC systems. “Groups using virtual machines will be able store more data inside of their virtual machines, closer to their custom application stacks.”

Comet is capable of an overall peak performance of almost two petaflops, or two million billion operations per second. It has the ability to perform 10,000 research jobs simultaneously. Like the tail of a comet, SDSC’s newest HPC cluster is intended to serve what’s called the ‘long tail’ of science: the idea that the large number of modest-sized computationally-based research projects represent, in aggregate, a tremendous amount of research that can yield scientific discovery.

“The goal of our partnership with SDSC is to inform the wider HPC community via papers and workshops on how to select the most appropriate, high-performance components suitable to their architectures and workloads, while gaining insight into how Seagate SSDs are used in domains that are relying on advanced computation and storage, such as genomics and the social sciences,” said Tony Afshary, director of ecosystem solutions and marketing for flash at Seagate.

The result of a National Science Foundation grant worth nearly $24 million including hardware and operating funds, Comet is available for use by U.S. academic researchers through the NSF’s eXtreme Science and Engineering Discovery Environment (XSEDE) program, a national collection advanced, integrated digital resources and services.

SDSC and Seagate will be discussing this collaboration at the upcoming Supercomputing 2015 (SC15) in Austin, TX, November 16^th through the 19^th. Please visit us at the SC15 exhibitor hall at booth #823. In addition, planning for the inaugural workshop on HPC architectures, best practices and SSD performance in operational HPC workloads will commence at the end of the year, with the first workshop scheduled for late spring or early summer 2016.

About Comet

Comet is a Dell-integrated cluster using Intel’s Xeon® Processor E5-2600 v3 family, with two processors per node and 12 cores per processor running at 2.5GHz. Each compute node has 128 GB (gigabytes) of traditional DRAM and 320 GB of local flash memory. Since Comet is designed to optimize capacity for modest-scale jobs, each rack of 72 nodes (1,728 cores) has a full bisection InfiniBand FDR interconnect from Mellanox, with a 4:1 over-subscription across the racks. There are 27 racks of these compute nodes, totaling 1,944 nodes or 46,656 cores.

In addition, Comet has four large-memory nodes, each with four 16-core sockets and 1.5 TB of memory, as well as 36 GPU nodes, each with four NVIDIA GPUs (graphic processing units).  The GPUs and large-memory nodes are for specific applications such as visualizations, molecular dynamics simulations, or de novo genome assembly.

About SDSC

As an Organized Research Unit of UC San Diego, SDSC is considered a leader in data-intensive computing and cyberinfrastructure, providing resources, services, and expertise to the national research community, including industry and academia. Cyberinfrastructure refers to an accessible, integrated network of computer-based resources and expertise, focused on accelerating scientific inquiry and discovery. SDSC supports hundreds of multidisciplinary programs spanning a wide variety of domains, from earth sciences and biology to astrophysics, bioinformatics, and health IT. SDSC’s Comet joins the Center’s data-intensive Gordon cluster, and are both part of the National Science Foundation’s XSEDE (eXtreme Science and Engineering Discovery Environment) program, the most advanced collection of integrated digital resources and services in the world.

About Seagate

Seagate is a world leader in storage solutions. Follow Seagate on Twitter, Facebook, Google +, YouTube, Instagram and subscribe to their blog.